Measuring Court Preferences, 1950 - 2011: Agendas, Polarity and Heterogeneity

Court scholars have a voracious appetite for Supreme Court preference measures. Several papers question whether widely-used Martin and Quinn scores provide valid intertemporal measures, calling into question virtually an entire generation of quantitative research on the Court. This paper discusses the challenges of inter-temporal preference estimation and revises, updates and extends Bailey and Maltzman (2011) to present Supreme Court preference estimates that are more defensibly comparable across time and institutions. ∗I appreciate comments from Albert Yoon. Any errors are mine. Court scholars have a voracious appetite for Supreme Court preference measures, whether in spatial or behavioral models of Court decision-making. While much of the demand for these measures has been filled by Martin and Quinn scores (Martin and Quinn 2002), not everyone is convinced that Martin and Quinn scores capture preference change over time. Some scholars note instances of implausible preference changes implied by the Martin and Quinn scores (Bailey 2007; Lauderdale and Clark 2012). Others, such as Ho and Quinn (2010, 846), note the strong assumptions required to use Martin and Quinn scores in a cardinal sense and encourage scholars to use the scores as ordinal rather than cardinal measures. The consequences for empirical Court research are substantial. Not only does Ho and Quinn’s advice affect the vast array of contemporary Court scholarship that uses Martin and Quinn scores, but it also raises two fundamental questions. Are Court preferences so difficult to estimate that we are doomed to using only ordinal, rather than cardinal, measures? Can anything be done to generate preference estimates that are comparable over time? The main challenge in generating inter-temporally comparable preference estimates is agenda change. Failing to distinguish agenda change from preference change can produce preference measures that conflate the two. For example, when we observe justices voting conservatively more often, is it because their ideologies have shifted? Or did this occur because the cases on which they were voting were different? These are questions we cannot answer by looking only at votes. Other challenges to generating accurate preference measures include identifying the ideological polarity of votes, modeling the functional form of preference dynamics and dealing with possible multidimensionality of preferences (Lauderdale and Clark 2012; Fischman and Law 2009).